Skip to content

gh-85943: Fix BytesWarning in the struct format cache under -bb#153627

Merged
serhiy-storchaka merged 1 commit into
python:mainfrom
serhiy-storchaka:fix-struct-cache-byteswarning
Jul 17, 2026
Merged

gh-85943: Fix BytesWarning in the struct format cache under -bb#153627
serhiy-storchaka merged 1 commit into
python:mainfrom
serhiy-storchaka:fix-struct-cache-byteswarning

Conversation

@serhiy-storchaka

Copy link
Copy Markdown
Member

The struct module caches compiled Struct objects in a dict keyed by the
format string. Since an ASCII str and the equal bytes object have the same
hash, using a str format after an equal bytes format (or vice versa) makes
the dict compare keys of different types, which raises BytesWarning under the
-bb command line option:

$ ./python -bb
>>> import struct
>>> struct.calcsize(b'!d')
8
>>> struct.calcsize('!d')
BytesWarning: Comparison between bytes and string

Fix it by decoding a bytes format to str (the same way Struct() already
does) before using it as the cache key, so equal str and bytes formats map
to a single entry and are never compared to each other.

Normalize bytes format strings to str before using them as the cache key,
so that equal str and bytes formats no longer collide and get compared.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@serhiy-storchaka serhiy-storchaka added needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes needs backport to 3.15 pre-release feature fixes, bugs and security fixes labels Jul 12, 2026
@serhiy-storchaka
serhiy-storchaka merged commit 190d2ff into python:main Jul 17, 2026
62 checks passed
@miss-islington-app

Copy link
Copy Markdown

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14, 3.15.
🐍🍒⛏🤖

@serhiy-storchaka
serhiy-storchaka deleted the fix-struct-cache-byteswarning branch July 17, 2026 07:49
@bedevere-app

bedevere-app Bot commented Jul 17, 2026

Copy link
Copy Markdown

GH-153834 is a backport of this pull request to the 3.15 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label Jul 17, 2026
@miss-islington-app

Copy link
Copy Markdown

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 190d2ffbb969ad571fd12d4988b37519091194d5 3.13

@bedevere-app

bedevere-app Bot commented Jul 17, 2026

Copy link
Copy Markdown

GH-153835 is a backport of this pull request to the 3.14 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Jul 17, 2026
@bedevere-app

bedevere-app Bot commented Jul 17, 2026

Copy link
Copy Markdown

GH-153836 is a backport of this pull request to the 3.13 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.13 bugs and security fixes label Jul 17, 2026
serhiy-storchaka added a commit that referenced this pull request Jul 17, 2026
GH-153627) (GH-153835)

Normalize bytes format strings to str before using them as the cache key,
so that equal str and bytes formats no longer collide and get compared.
(cherry picked from commit 190d2ff)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
serhiy-storchaka added a commit that referenced this pull request Jul 17, 2026
GH-153627) (GH-153834)

Normalize bytes format strings to str before using them as the cache key,
so that equal str and bytes formats no longer collide and get compared.
(cherry picked from commit 190d2ff)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
serhiy-storchaka added a commit that referenced this pull request Jul 17, 2026
GH-153627) (GH-153836)

Normalize bytes format strings to str before using them as the cache key,
so that equal str and bytes formats no longer collide and get compared.


(cherry picked from commit 190d2ff)

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant